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EDITORIAL 
EVALUATING ROENTGENOGRAPHIC TECHNIQUES 


The leading article in this issue, “Statistical Problems in Assessing 
Methods of Medical Diagnosis with Special Reference to X-ray 
Techniques,” by Jacob Yerushalmy, is an extension of a study 
which originally was published in the Journal of the American Medical 
Association.! This study is regarded as a striking contribution to 
knowledge in the field of roentgenology and has given rise, in some 
quarters, to a dissent not unrelated to apprehension. Upon first 
consideration, the conclusions of the original article, as well as those 
of the present one, may appear to arraign the roentgenologist and to 
question the validity of his techniques. Closer and more impersonal 
scrutiny, however, permits two positive and optimistic observations, 
the truth of which gives impetus to future corrective action. These 
are: (a) there is no significant difference in the effectiveness of the 
various sizes of X-ray film, and (b) other areas of intellectual discipline 
entertain similar imperfections of technique. The first observation 
permits confidence in the application of case-finding techniques; the 
second destroys infertile complacency and prepares the soil for eventual 
growth. Unlike an Achilles heel, the discrepancies cited in these 
studies are in no sense indigenous defects but phenomena susceptible 
to remedy. 

When mass radiography was first employed as a technique of tuber- 
culosis case finding, certain disputations denied the effectiveness of 
miniature film in the detection of abnormalities within the pulmonary 
parenchyma. The current studies present beyond possibility of 
denial that no one size of film, not even the 14’’ x 17”’ celluloid, is 
superior to any other size in effectiveness for case finding. The prob- 
“1 Birkelo, Carl C.; Chamberlain, W. Edward; Phelps, Paul S.; Schools, Percy E.; Zacks, David; and 
Yerushalmy, Jacob: A comparison of the Effectiveness of Various Roentgenographic and Photofluorographic 
Methods, J. Am. Med. Assoc., 133: 359-365 (1947). 

*This is the twentieth of a series of special issues of PUBLIC HEALTH REPORTs devoted exclusively to 
tuberculosis control, which will appear the first week of each month. The series began with the Mar. 1, 
1946, issue. The articles in these special issues are reprinted as extracts from the PUBLIC HEALTH REPORTS. 
Effective with the July 5, 1946, issue, these extracts may be purchased from the Superintendent of Docu- 


ments, Government Printing Office, Washirigton 25, D. C., for 10 cents a single copy. Subscriptions are 
obtainable at $1.00 per year; $1.25 foreign. 
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lem here is not film quality or size but film reading. When, upon 
two readings of a series of films, few readers can be consistent with 
themselves or with others, further investigation of the causes of such 
variation is imperative. This so-called personal equation which oper- 
ates as an error-factor in film interpretation demands the most serious 
consideration. Until the problem is solved, it has been suggested 
that all survey films be read independently by at least two interpreters. 
Furthermore, an evaluation of teaching methods employed in depart- 
ments of roentgenology might suggest profitable changes that would 
produce improvement in roentgenologic accuracy. Certainly, aware- 
ness of error is the threshold to correction. 

X-ray examination of the chest, however, compares favorably with 
any other diagnostic method in the field of medicine, and it must be 
pointed out with emphasis that the lesions interpreted in these studies 
are subtle in character and minimal in extent. Almost all gross 
pathology is visualized on the roentgenogram. Since the great ma- 
jority of lesions observed in mass X-ray surveys are minimal, these 
studies of abstruse lesions assume added significance. 

The field of roentgenology is fortunate in that the evidence produced 
by its material and tools is measurable and demonstrable. There are 
many fields of knowledge in which error is known to exist, but so 
intangible is the evidence, that the magnitude of that error cannot 
be demonstrated. It remains for us now to continue our investiga- 
tions and seek out the approaches to perfection. 

Francis J. WEBER, 
Medical Director Chief, Tuberculosis Control Division. 


STATISTICAL PROBLEMS IN ASSESSING METHODS OF 
MEDICAL DIAGNOSIS, WITH SPECIAL REFERENCE TO 
X-RAY TECHNIQUES ' 

By Jacos YervusHaumy, Ph. D. 

The process of medical diagnosis involves the application to a 
specific case of the knowledge accumulated from a large number of 
similar cases. This knowledge may have been derived by systematic 
observation and detailed analysis of case records, or it may have been 
obtained intuitively in the course of the physician’s experience. In 
either instance the process is statistical in nature in that it consists of 
abstracting trom a multiplicity of factors and conditions those which 
are pertinent to specific cases. Moreover, since no two cases are 
exactly alike, the resulting diagnoses are not absolute but involve some 
uncertainty and might better be thought of in terms of probabilities. 
These probabilities may, in some instances, be very high; in others 

1 From Field Studies Section, Tuberculosis Control Division. This paper and the discussion by Professor 

Neyman, which follows it, were presented before the Institute of Mathematical Statistics, at the twenty- 

eighth annual meeting of the Pacific Division of the American Association for the Advancement of Science, 


in San Diego, Calif., June 18, 1947, while the author was visiting professor of biostatistics at the School of 
Public Health, University of California. 
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they may have lower values. One of the fundamental objectives of 
improved medical care is to increase the probabilities of correct 
diagnoses. 

This objective is being accomplished by the increasing employment 
of various diagnostic aids such as X-ray and laboratory investigations. 
However, even with the best techniques it is not always possible to 
raise the level of diagnosis to absolute certainty in all cases of a certain 
disease. Medical diagnosis of an individual case is often assisted also 
by the fact that it is not based on a single observation. Repeated 
studies of the same case allow for integration of a number of related 
observations and for continuous revision of tentative diagnoses. The 
chance of error is thus reduced ang the probability of correct diagnosis 
is increased. 

There are, however, occasions when the benefit of repeated and 
continuous case studies is lacking, and at least a preliminary diagnosis 
must be made on the basis of a single observation. This is especially 
true in the field of public health when examinations for specific pur- 
poses are made on relatively large samples of the population. Two 
main types of public-health activity may be noted where the “single 
event” form of diagnosis is employed. 

In problems of the first type, the information required is a knowledge 
of the total number of positive* persons in the group but not the 
identities of the individual positive cases. In activities of the second 
type the main purpose is the identification of the persons in the 
population having a certain characteristic. 

Examples of activities of the first type are mass examinations of 
population groups for the purpose of determining an epidemiological 
index for some infection or other pathological condition. All that is 
required in such mass examinations is a knowledge of the population 
examined and the total number of “‘positives.’”’ Because the indi- 
duals need not be identified, the numerical value of the index may 
in some instances approach more nearly the true value, because the 
“false negatives” (positive individuals who have been diagnosed 
“negative’’) are compensated to a certain extent by “false positives”’ 
(negative individuals who have been diagnosed ‘‘positive’’). 

Examples of activities of the second type are X-ray examinations 
or serological tests conducted on large population groups as control 
measures in tuberculosis or syphilis. In these screening examinations 
the main objective is not to ascertain the total number of “‘positives’”’ 
in the sample but to detect the persons possessing certain character- 
istics as a first step for further examinations and action on these 
“positive” cases. 

Another type of activity in which the diagnosis is based on a single 


? The term “positive” refers to individuals who possess the characteristic to be detected by the examina- 
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observation relates to studies on the efficacy of different diagnostic 
aids. Also in this instance it is not sufficient to know how many 
persons in the sample are positive but it is also necessary to identify 
the persons who possess certain characteristics. For example, if it is 
desired to compare the performance of different tests for syphilis in 
detecting infected individuals, it becomes necessary to test the same 
sample of the population by each of the different methods and to 
compare the findings for each individual of the sample. 

The task of such comparative studies, when approached by the 
investigator, presents a serious dilemma. If an absolute comparison 
between the different diagnostic aids is to be made, it becomes neces- 
sary to identify the individuals in the sample who are positive. How- 
ever, this identification can be accomplished only through the use of 
one or the other of the tests. It is, in a sense, a vicious circle. 

The general procedure in dealing with this type of problem is to 
select one of the techniques as a “standard”? and to compare the 
performance of the other techniques with that of the standard. This 
solution is not entirely satisfactory in many instances. For one thing 
such a procedure implies that the diagnosis provided by the standard 
technique is correct in altinstances. In addition, it delays the recogni- 
tion of the superiority of a technique which may be better than the 
accepted standard. 

It is, therefore, desirable in such problems as were mentioned 
above, as in others of similar nature, to devise methods of examination 
and analysis which would increase the probabilities of correct diagnosis 
in mass examinations, where the subjects appear for examination 
only once. One obvious method is to substitute multiple observations 
on the single examination for part of the benefits which could be 
derived if the subjects were available for study. This may be accom- 
plished by the simultaneous employment of a number of related tests 
or by increasing the number of independent observers or by a com- 
bination of both of these methods. 

It is the object of this paper to utilize data from a comparative 
X-ray study in an attempt to develop a method of defining ‘‘positive’”’ 
cases in the test population which is not dependent on any one 
diagnostic technique or on the interpretations of any single observer. 
Instead, the information yielded by all the techniques and all observers 
is used to increase the probabilities of correct diagnosis and thus 
furnish a more objective method of comparing the different X-ray 


techniques. 
MATERIAL 


The material for this study is provided by an investigation on the 
relative effectiveness for tuberculosis case finding, of various photo- 
fluorographic and roentgenographic methods. 
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Some 1,200 persons comprising the entire population of a Veterans’ 
Administration institution were X-rayed consecutively on four differ- 
ent X-ray techniques providing for each person a 35-mm. photofluoro- 
gram of the chest, a 4’’ x 10’’ stereophotofluorogram, a roentgenogram 
on 14’ x 17’’ paper negative, and a conventional 14’’ x 17’’ celluloid 
film. 

These examinations provided, therefore, four sets of chest films of 
different size on the same group of individuals taken at the same time. 
These sets of films were interpreted independently by five expert radi- 
ologists and chest specialists. A more detailed account of the method 
of interpretation is given in a previous report.’ For present purposes 
it is sufficient to mention that a number of meetings of the readers 
were held before the films were circularized, a method of interpretation 
was developed and experimented with, in order that the five readers 
reach as nearly as possible uniformity in nomenclature. 

The films were sent to the readers one set at a time. At the com- 
pletion of interpretation of all four sets and after a lapse of from 2 
to 3 months, the 14’ x}17’’ celluloid films were again circularized 
among the readers for a second independent interpretation of the same 
films. These activities yielded 25 sets of independent interpretations 
of the four sets of films, which constitutes the material available for 
analysis. 

ANALYSIS 


The object of the analysis is to obtain measures for the relative 
efficiency of the different X-ray techniques in selecting the individuals 
in the study group who have X-ray evidence suggestive of tubercu- 
losis. The usual procedure is to determine for each of the four tech- 
niques two measures: . 

1. A measure of sensitivity or the probability of correct diagnosis 
of “positive” ‘ cases, and 

2. A measure of specificity or the probability of correct diagnosis 
of “‘negative’’ cases. 

The determination of these measures is complicated by two main 
difficulties in analysis which are present in many comparative studies 
of this kind. The first results from the fact that it is not known who 
in the test population is positive and who is negative, i. e., who should 
be selected in the screening test for further study because of X-ray 
evidence suggestiveof tuberculosis and who is free from such evidence. 
The second difficulty is due to subjective errors of interpretation. 
The latter are of two main types: interindividuals, or the inconsistency 


+ Birkelo, C. C., Chamberlain, W. E., Phelps. P. 8., Schools, P. E., Zacks, D. and Yerushalmy, J.: 
Tuberculosis Case Finding. J. Am. Med. Assoc., 133: 359-365 (1947). 

‘ By “positive” cases are meant persons who have X-ray evidence suggestive of tuberculosis, and “‘nega- 
tive” cases refer to persons who have no such evidence. 
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of interpretation found among different readers and intraindividuals, or 
the failure of a reader to be consistent with himself in independent 
interpretations of the same set of films. If these subjective errors 
were small in magnitude, they would cause little difficulty. In the 
present study, however, they were relatively large. For example, even 
with the 14’’ x 17” celluloid technique, which is usually taken as a 
standard, the number of “positives” selected from among the 1,256 
films was as low as 56 for one reader and as high as 100 for another 
with intermediate numbers for the other three readers. In other 
words, not only is it necessary to select a “standard technique” but 
also a “standard reader.” 


That these difficulties may lead to wrong conclusions may be seen 


by analyzing the material, as is so often done, without reference to 
these problems. This was demonstrated in the main report of the 
study * in the following way. The 14’ x 17’’ celluloid technique was 
taken as a “standard” and the interpretations of only one expert 
were considered. Reader N selected on the standard technique 59 
cases as showing X-ray evidence suggestive of tuberculosis. Of these, 
he failed to diagnose tuberculosis in 27 percent of the corresponding 
35-mm. films, in 30 percent of the 4’’ x 10’ films and in 24 percent 
of the 14’’ x 17”’ paper negatives. 

From the performance of reader N, one would be tempted to drew 
two conclusions: first, that none of the test techniques can be con- 
sidered efficient (the lowest percentage missed is 24 percent) and, 
second, that if a choice must be made, the 14’’ x 17’’ paper negatives 
have an edge over the two miniature techniques. | 

It was shown, however, that neither of these conclusions is justified. 
For, first, the same reader N missed on his first reading of the 14’’ x 
17’’ celluloid films 22 percent of:those which he called positive for 
tuberculosis on the second reading of the same 14’’ x 17’’ celluloid 
films. In other words, the relatively high percentages of missing posi- 
tive cases do not necessarily measure the limitations of the different 
techniques, but, to a large extent, the subjective errors of interpreta- 
tion for reader N. 

Secondly, it is not possible to conclude that the paper negatives 
are more reliable than the miniature techniques because the same 
results are not found for all the readers. For reader M the score on 
the 4”’ x 5”’ is better than on the other two. For readers O and Q 
the best performance was that on the 35 mm., while reader P had the 
same score on all three techniques. 

This example illustrates that no valid conclusions may be drawn 
from an analysis which fails to take into account the subjective errors 


5 Loc. cit. p. 361. 
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of interpretation. Even if the 14’’ x 17” celluloid film is taken as a 
standard, it becomes necessary to resolve the variance found for the 
different techniques into its components and to minimize the effect 
of the subjective errors. 

One method which accomplishes this end result consists in basing 
the analysis on “group opinion” derived from the individual inde- 
pendent interpretations. Briefly, the procedure is to consider as 
“positive” only those cases which were so called on the standard 
technique by more than one interpreter. Since the interpretations 
were all independent, the probability that a positive case will be missed 
by more than one reader is greatly reduced. Similarly, it is unlikely 
that a 14’’ x 17” film which shows no X-ray evidence suggestive of 
tuberculosis will be called positive by several readers, each reading 
independently of the others. 

By this method it is possible to line up the three test techniques and 
to compare their performances to that of the standard 14” x 17” 
celluloid technique. For example, there were 61 cases which were 
called positive on the 14’’ x 17’ celluloid by three or more inter- 
preters. These are considered the only ‘‘positive’’ cases in the popu- 
lation examined, i. e., these 61 are taken to be the only ones who pre- 
sumably have X-ray evidence of tuberculosis and who should be 
selected, by a screening technique, for further study. The films for 
these 61 cases are then reviewed for each of the other techniques, and 
a technique is considered to have missed one of these cases if at least 
3 of the 5 interpreters called it negative for tuberculosis. The result 
of the comparison by this method in the present study was that each 
of the test techniques missed the same number of films (10 percent). 

This method eliminates the major part of the difficulty resulting 
from subjective errors of interpretation, but retains the limitation that 
one of the techniques (in this case the 14’’ x 17’’ celluloid) is taken as 
a standard. What may be concluded is that the 35 mm., the 4’’ x 
10’, and the 14’”’ x 17’’ paper techniques are equally efficient in 
selecting positive cases. However, the impression obtained from the 
above that the efficiency of these techniques is approximately 90 
percent that of the 14’’ x 17” celluloid technique cannot be accepted 
without further study. 


COMPARISON OF ALL FOUR TECHNIQUES WITHOUT REFERENCE TO A 
STANDARD 


It now becomes desirable, if possible, to remove the last restriction, 
that of selecting the 14’’ x 17’’ celluloid technique as a standard, and 
to devise means whereby the four techniques may be compared on an 
equal footing. 

When the problem of comparing the four techniques is approached 

760653—47——2 
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without reference to a predetermined standard, it is useful to dis- 
tinguish between false information inherent in the technique and false 
information due to incorrect interpretation by a reader. A shadow, 
for example, may be very definite on the technique but missed or mis- 
interpreted by a reader. On the other hand, a distinct shadow which 
suggests disease may be present on a film, but in reality the person has 
no disease and the shadow on the film is due to an artefact or to some 
exaggerated markings; again, a shadow of an existing lesion may not 
appear on the film or appear so indistinctly that it cannot be clearly 
visualized. It is the existence of those latter types of false information 
which argues against the selection of any technique as a standard. 

In order to visualize more clearly the interplay of the two types of 
false information mentioned, it is useful to consider how they are 
affected by the addition of more techniques and more observers. 
Suppose, first, there were only two techniques and one reader. 
Each person examined can then be represented by a twofold table in 
which the first symbol is the interpretation of the reader on one of the 
techniques and the second represents the interpretation of the same 
reader on the second technique. Each person will be represented by 
one of the following four types of tables: 


TYPES 





> 


Technique | 1 | 2 | 3 | 4 


Sette P=t , dF | rr oases 








Persons represented by tables of the first two types are, on the avail- 
able evidence, clearly defined. Those of type 1 are probably negative 
and those of type 2 are probably positive. The status of persons 
represented by tables of type 3 and 4, however, is undetermined. If 
the two techniques are given equal weight, it is impossible to say 
whether the persons are positive or negative. In this case it becomes 
necessary to select one of the techniques (for example that repre- 
sented by the first symbol) as a standard. Persons represented by, 
tables of type 3 are then considered “‘negative,’’ and those represented 
by table 4, “positive.” The false information is then attributed 
entirely to the test technique. The positive reading in a table of type 
3 is considered a “false positive,” and the negative reading in a table 
of type 4 is considered a ‘false negative’ and counted against the 
performance of the test technique. 

Suppose now that another interpretation is obtained on the same 
two sets of films. Each person in the study may now be represented 
by a fourfold table in which the symbols in the first row represent 
the two interpretations on one technique, and those in the second 
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row the interpretations on the second technique. There will now be 
16 distinct types of cases. For the present purposes, however, it 
is sufficient to focus attention only on those cases which previously 
were represented by tables of type 3 and 4. Among these there will 
be some in which the second reading on the first technique will con- 
firm the original reading on the second technique rather than that of 
the first technique. For example, a case which originally was of type 


“_|may take the form of], Tl. In other words, a -case 
+ + + 





3, 1¢., 
which was originally read negative by technique S and positive by 
technique T, is read as positive on technique S on the second read- 
ing. It is obvious that this case has a very strong probability of 
being positive, and therefore it should have originally been counted 
as a “false negative” for the standard technique rather than a “false 
positive” for the test technique. Again a case which originally was 
of type 4,| t bagee! 
nally have been counted a “false positive” on the standard rather 
than a ‘“‘false negative” on the test technique. 

The addition of a second reading, therefore, undermines our con- 
fidence in the simple procedure used when only one reading was 
available in which all the false information was thought to be derived 
from the test technique. It is now apparent that a certain part of 
it must be attributed to the standard. It should be noted than this 
modification results from the presence of subjective errors attributed 
to the reader. 

Although the addition of the second technique resolves the un- 
certainty of some of the cases, there still remain undetermined cases 


This case should have origi- 





may now become 


of the form| w + land | 4 4 | . For these, the evidence provided by 


the two techniques is conflicting, but the interpretations are consistent 
for each technique. If no additional evidence is available, the un- 
certainty can be resolved only by again considering one of the tech- 
niques as standard (the technique represented by the two readings 
in the first row) and the other as a test technique. In other words, 
persons represented by tables of the first type (those read positive 
on the standard) are considered to be “positive,” while those repre- 
sented by tables of the second type (read negative on the standard) 
are considered ‘‘negative.”” The number of cases of the first type . 
will, therefore, furnish a measure of lack of sensitivity of the test 
technique, while those of the second type will indicate its lack of 
specificity. It should be noted that since the two interpretations for 
each technique were in agreement, the discrepancies are more nearly 
the results of differences in the techniques themselves and not of 
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subjective or human errors of interpretation. On the available 
evidence all disagreements are counted against the test technique. 
Suppose, however, that a third technique is introduced, i.e., every 
person has been tested by three different techniques and the films of 
each technique have been interpreted twice, either by the same or 
different readers. The persons in the study can now be represented 
by 64 different types of tables of 3 rows and 2 columns. Again, at- 
tention may be focused only on the undetermined cases which were 
+ eI 


previously represented by tables of the form|* and rm" +4 ry Some 


+4 


++]. In other words, the 
, 
readings on the third technique confirm not those of the standard 


(first row), but they agree with the readings of the test technique 
(second row). 

The weight which may be assigned to this additional information 
as evidence as to whether the cases are likely to be positive cr negative 
may be debatable. To some, the superiority of the standard as 
compared to any test technique is so completely accepted that it 
outweighs the combined evidence provided by any number of other 
techniques. To others, the fact that two different films confirm one 
another as to the presence or absence of a shadow will carry sufficient 
weight to cast doubt on the evidence provided by the standard. It 
is probably not too much to ask of even the confirmed believer in 
the standard to review again the films for these cases. In actual 
practice such a review reveals frequently that the combined evidence 
of the test techniques more nearly represents the probably true facts 
than does the evidence provided by the standard. This is under- 
standable because the errors responsible for the types of discrepancies 
represented by such tables as are here discussed are traceable not to 
interpretation but to such considerations as positioning and physical 
characteristics of the films. It is, for example, more likely that a 
shadow will be hidden behind a rib in one film than that a shadow 
of a non-existent lesion would occur twice on two different films. 
Indeed, it is the exceptional case where the evidence provided by a 
single technique comes nearer to revealing the true facts than does 
that provided by two or more techniques. 

From the above it is seen that the addition of a technique or of a 
reader modifies our confidence in the infallibility of the standard. 
The modification from the addition of a reader is associated with the 
presence of subjective errors, while that from the addition of another 
technique is mainly a result of errors which are inherent in the tech- 
nique. When both the number of readers and the number of tech- 


of these may now become and 
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niques are increased, additional information becomes available which 
may be used to evaluate for specific cases the probability of their 
being positive or negative. Although it is not possible to assign 
a priori relative weights to the evidence provided by the different 
techniques and the different readers, the utilization of all the informe- 
tion provided by them makes possible a more comprehensive analysis 
of the material, which may lead to a more realistic evaluation of the 
relative efficiency of the different techniques. 


OBJECTIVE COMPARISON OF THE FOUR TECHNIQUES 


If, as a first step, it is assumed that each technique and each reader 
has equal weight as evidence that the case is either positive or nega- 
tive, the problem of comparing the different techniques becomes rela- 
tively simple. The definition of a “positive” individual can then be 
made to depend on the number of positive readings obtained for him 
on all the techniques by all the readers. In the present study, for 
example, every person may be represented by a table of 4 rows (4 
techniques) and 5 columns (5 readers), thus providing for each person 
20 different readings. The individuals can be classified in 21 groups 
depending on the number of positive readings, ranging from 0 to 20. 
The probability that the last group is positive is obviously very great. 
Similarly, individuals with all 20 readings negative are, on X-ray evi- 
dence, negative for tuberculosis. Cases with one or only a very few 
positive readings are also very likely negative. As the number of 
positive readings increases, it becomes more difficult to determine 
where “false positive” stops and “true positive” begins. The decision 
as to the dividing line between “positive” and “negative’’ must, of 
necessity, be arbitrary. Some may want a relatively large number of 
positive readings before considering an individual positive; others 
may want to investigate individuals with a relatively small number 
of positive readings. The important thing, however, is that once the 
number is decided upon, it becomes a simple matter to compare the 
ability of the four techniques in selecting these “positive” ¢ases. All 
that. is necessary is to review X-ray interpretations for all individuals 
who are “positive” by the accepted definition and to count the num- 
ber of positive readings yielded by each technique. A technique 
which is more sensitive should yield a larger number of positive read- 
ings on these cases than a less sensitive technique.* 

This procedure has been followed in the present study, and a com- 
parison has been made in different definitions of ‘‘positive,” all yield- 
ing similar results. (See table 1.) For example, on the basis of a 
majority (11 or more) of the 20 readings, there were in the present 





* This discussion limited tq an investigation of the sensitivity of the different techniques. The problem 
of specificity will be considered in ancther publication. It may be seen from table 1 that there are differences 
in specificity among the four techniques. 
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study 62 “‘positives.”” The maximum number of positive readings 
that could be obtained on any one technique for these “‘positive”’ 
cases is 62 x 5, or 310. The actual number of positive readings was 
259 for each of the three test techniques and 250 for the 14’’ x 17” 
celluloid. Similar results are obtained when “‘positive”’ is defined by 
six or more positive readings. 
















TABLE 1.—Number of readings called positive by all readers on each technique 
when the cases are classified according to the total number of positive readings by 
all readers on all techniques 


























Maximum | Number of positive readings by all 
possible readers on— 
Total number of positive readings by | Number ee —— 
all readers on all techniques of cases ee 
gs on 4" x10” | 14% x17” | 14" x17” 
any one 35-mm. ; 
technique stereo paper | celluloid 
iI ccechccnaieh aldtsieaiiidnniagihintninieeeictichtniadnimane 171 855 109 90 49 50 
Z Se ee RE 145 54 54 58 47 
EP SSS ae 12 60 36 36 37 37 
RR ARR I ATE ie Gt FP 20 100 179 76 75 69 
itt tannin cdsttdainitlendibeainaidempeeinsiai y 45 39 42 42 39 
20 105 105 105 105 105 
SE ee a eee 41 205 154 154 154 145 
6-19... 60 350 199 199 202 192 























If the assumption of equal weight for each technique and each reader 
is valid, it would be possible to conclude from the above that there is 
little to choose between the four techniques and that they are all 
equally efficient in finding cases of tuberculosis in mass survey work. 













TESTS FOR THE VALIDITY OF THE METHOD 





It is now necessary to investigate the possible limitations of this 
method of analysis. In the main, the testing must determine whether 
the method is likely to include an appreciable number of negative 
cases among those that are defined as “positive” or to include a number 
of positive cases among the “‘negatives.”’ ; 
As to the latter, it is possible that.the “negative” group contains a 
number of important positive cases which by the accident of definition 
were not included among the “positives.’”” There may, for example, 
be a certain type of lesion which one of the techniques, say the 14’’ x 
17’’ celluloid, detects but which the other techniques fail to identify. 
Such a case may even have as many as 5 positive readings on the 14’ 
x 17’ celluloid film, but will have fewer than 11 total positive readings 
and wil therefore be included among the “negatives.”” This ob- 
viously is an extreme case, but a tendency in this direction, even though 
less pronounced, may nevertheless lead to the exclusion of a number 
of significant cases from the group of “positives.” 
Among the cases which are “positive” by definition, the method 
may also fail because of a tendency to overread, which may operate 
on the test techniques. A technique may accumulate as many 
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positive readings on the “positive’’ cases as another technique, not 
because it is equally sensitive but as a result of overreading on some 
of the films and underreading on others. It will make up by over- 
reading what is missed by underreading and have the total number of 
positive readings approximately equal to that of the other techniques. 

Whether either of these tendencies exists can be tested by comparing 
the distributions of the films according to the number of positive 
readings for each of the different techniques. If it is true that the 
group of “negatives” contains an appreciable number of positive 
cases which were selected by the 14’’ x 17’’ celluloid but which failed 
to be included in the “‘positive’’ group because the other techniques 
failed to select them, there would be among the “negatives’’ more 
cases with three or more positive readings for the 14’’ x 17’’ celluloid 
than for-any of the others. Similarly, if there is a tendency on the 
part of a test technique to overread some and underread others of the 
“positive” films, then the distribution for that technique should be 
in the form of a U-shaped curve, with many cases having zero or one 
positive reading, and many with four or five positive readings while 
cases with an intermediate number of readings would be relatively 
few. 

When the distributions for the different techniques are compared 
separately on the “negative” cases and the “positive’’ cases, (see 
tables 2a, 2b and figs. 1 and 2), it is seen that they are similar for the 


TaBLe 2a.—Distribution of the 29 cases with 6-10 positive readings on all tech- 
niques according to the number of positive readings on each technique 
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TABLE 2b.—Distribution of the 41 cases with 11-19 positive readings on all tech- 
niques according to the number of positive readings on each technique 





Number of positive readings 
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four techniques and that such variations as exist from technique to 
technique are not of the type which would indicate that the limitations 
mentioned above exist. 

It should be pointed out that there undoubtedly are a number of 
persons who probably have X-ray evidence of tuberculosis but who 
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are excluded from the group of “positive” cases (11 or more positive 
readings) because of the failure of one or more of the techniques to 
identify the lesion on the corresponding films. These persons may 
be found mainly in the group with less than 11 and more than 5 
positive readings. The important consideration, however, is whether 
this happens more often with one technique than with another. A 
second test that needs to be made, therefore, consists in the following: 
If the 4 techniques contribute equal numbers of positive readings 












FIGURE 1. DISTRIBUTION OF THE 29 CASES WITH 6 - 10 POSITIVE 
READINGS ON ALL TECHNIQUES ACCORDING TO THE NUMBER OF 
POSITIVE READINGS ON EACH TECHNIQUE. 
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when “positive” cases are defined in terms of a majority of positive 
readings (11 or more), the same should hold true if the definition of 
positive cases is broadened to include borderline cases with perhaps 
6 or more positive readings. In the present study this was the case. 
(See table 1.) 

In addition, it is necessary to review carefully each of the cases 
which may have been excluded from the “positive” group: (11 or 
more positive readings) by virtue of the failure of one or more of the 
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techniques. The number of such cases and their distribution for the 
four techniques is shown in table 3. In this table are shown all the 
cases for which there were at least 3 positive readings recorded on 1 of 
the 4 techniques but for which the total number of positive readings 
was less than 11. 


2. DISTRIBUTION OF THE 41 CASES WITH 11-19 POSITIVE 
READINGS ON ALL TECHNIQUES ACCORDING TO THE NUMBER OF 
POSITIVE READINGS ON EACH TECHNIQUE. 
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NUMBER OF POSITIVE READINGS ON EACH TECHNIQUE 


It is important to note, first, that each of the 4 techniques can claim 
i number of possibly positive cases which were excluded from the 
group of 11 or more positive readings by virtue of the failure of the 
other techniques. Actually there were fewer such cases which the 
14’ x 17’ celluloid technique detected than there were for each of 
the other techniques. In view of the possibility, however, that 
‘“‘positives’”’ detected by the 14’’ x 17’ celluloid technique may be 
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more likely to be positives than those detected by the other tech- 
niques, it is necessary to investigate in more detail the 4 cases in the 
“negative” group (less than 11 positive readings) which have 3 or 
more positive readings on the 14’’ x 17’’ celluloid films. Of these, 1 
had only 3 additional positive readings on the other techniques (out 
of a total of 15), 2 had 5 additional positive readings each, and 1 had 
7. It is likely that some, if not all, of these cases may actually repre- 
sent positive individuals. It is, however, remarkable that even with 
the small amount of material available in this study, it was possible to 
find four other cases which were almost identical to these four, except 
that the 14’’ x 17’’ celluloid readings are interchanged with one of the 
test techniques. 

Table 3.—Number of additional positive readings on the other techniques for cases 


which were called positive by 3 or more readers on a specified technique for all cases 
with less than 11 positive readings on all techniques 





Number of additional positive readings on other 

Num- techni jues 

Called positive by three or more readers on— = of 
ases 






























The actual readings on all the techniques for these two sets of four 
cases are presented in table 4. A review of these tables indicates 
similar situations for the 14’’ x 17’’ celluloid and for the other tech- 
niques. For example, case number 542 had four positive readings on 
the 14’’ x 17” celluloid, three positive readings on the 35 mm., two on 
the 4’’ x 10’’ stereo, and no positive readings on the 14’’ x 17’’ paper. 
It is almost certain that the individual represented by this table is 
positive and that this case is not included in the “positive’”’ group (11 
or more positive readings) because of the failure of the 14’’ x 17’’ paper 
technique to select it. However, case number 939 shows almost an 
identical picture except that the paper recorded four positive readings 
while the celluloid failed to identify it. When the two sets of four 
films for these two cases were reviewed by the radiologists, it was 
agreed that both represent positive cases, and the failure of the specific 
technique to select them was assigned to the physical characteristics 
of the film or to the positioning of the patient. The important con- 
sideration is whether these technical or chance misses happen more 
frequently with the test techniques than with the standard. In the 
present study, as was shown above, this was not the case. 

It may also be of interest to supplement the above test by means of 
a hypothetical binomial distribution. The observed distribution 
according to the number of positive readings may be compared with a 
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theoretical one which is obtained from the assumption that all the 
techniques are equally sensitive and that the probability of missing a 
positive film is constant for the five readers. While this test can be 
made on the entire distribution, including also the probability of miss- 
ing a negative film, the methodology becomes too complicated. For 
the present purposes it was thought sufficient to test only the sensi- 
tivity of the different techniques, since in practical work sensitivity 
is more important than specificity. The test can, therefore, be made 
only on that part of the distribution embracing 11 or more positive 
readings. Furthermore, there was a relatively large number of cases 
(21) in which all 20 readings were positive. It was obvious from a 
knowledge of the approximate probability of missing a positive case 
_ (higher than 0.20) that no binomial distribution which would approxi- 
mate the rest of the distribution would yield as many as 21 cases with 
20 positive readings. It was therefore necessary to modify the 
hypothesis and to assume that there are a certain number of cases 
with very obvious lesions (approximately 21) whose probability of 
detection is 1, i. e., that no technique and no reader will miss them. 
It is then possible to compare the theoretical distribution obtained 
from the binomial with that part of the observed distribution ranging 
from 11 positive readings to 19. The average probability of missing a 
lesion on these cases was found to be .276. A distribution given by 
N(.276+.724)” will, therefore, test the hypothesis that with the ex- 
ception of some 21 cases whose lesions are so obvious that no reader 
and no technique will miss them, there is an equal probability for each 
of the techniques and each of the readers to miss the other “‘positive”’ 
cases. The value of N which yielded the smallest value of x? was 41. 


TaBie 5.—Chi Square test for goodness o . “fit — the theoretical values derived 
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from the expansion of the point binomina f 41 *(0.276 + 0.724) 
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The expected frequencies according to this binomial distribution 
and the observed frequencies are shown in table 5. It may be seen 
that the fit is not too bad, the chi-square test gives for P the value 
0.18. In other words, the test provides no reason for rejecting the 
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hypothesis. Incidentally the value of 41 obtained for N together 
with the 21 cases with 20 positive readings yielded a total of 62 positive 
cases, or the same number obtained by using 11 or more positive read- 
ings according to the arbitrary definition adopted. 


SUMMARY AND CONCLUSIONS 


In this paper an attempt is made to compare the effectiveness, for 
tuberculosis case finding, of various photofluorographic and roent- 
genographic methods. Some 1,200 individuals were examined con- 
secutively on four different machines yielding for each a 35-mm. 
photofluorogram, a 4’’ x 10’’ stereophotofluorogram, a roentgenogram 
on 14’’ x 17’ paper negatives and a 14’’ x 17”’ celluloid film. These 
were interpreted independently by five expert radiologists and chest 
specialists. A second independent interpretation was obtained from 
each reader on the 14” x 17” celluloid films. 

A method was devised by which the comparison was made objec- 
tively without a predetermination of any of the techniques as a stand- 
ard. Instead, the evidence yielded by all five readers on all four 
techniques was utilized in defining “ positive” cases, and a comparison 
of the four techniques was based on their ability to detect these ‘ posi- 
tive” cases. Several tests of the validity of this method are presented. 

The results of this analysis justify the conclusion that, strictly 
from the point of view of their ability to find cases of tuberculosis in 
mass survey work, none of the techniques, not even the 14’’ x 17’’ 
celluloid, is superior to any of the others. 


OUTLINE OF STATISTICAL TREATMENT OF THE PROBLEM 
OF DIAGNOSIS ' 


By J. NeyMAn, Professor of Mathematics and Director of the Statistical Laboratory 
University of California, Berkeley, California 


(1) 


Every attempt at a mathematical treatment of phenomena must 
begin by building a simplified mathematical model of the phenomena. 
In studying the problem of diagnosis as presented by Dr. Yerushalmy,? 
we have to consider a population (or a universe) U of individuals 
which we shall imagine divided into three exclusive categories. 

(i) individuals entirely free from the given disease, whose 
proportion is a. 

(ii) Individuals moderately affected by the disease, whose pro- 
portion is 8. 


' This paper was presented before the Institute of Mathematical Statistics, at the twenty-eighth annual 
meeting of the Pacific Division of the American Association for the Advancement of Science, in Sen Diego, 
California, June 18, 1947 as a discussion of the foregoing paper by Jacob Yerushalmy. 

? See “Statistical Problems in Assessing Methods of Medical Diagnosis, with Special Reference to X-ray 
Techniques” by Jacob Yerushalmy, this issue. 3 














October 8, 1947 1450 





(iii) Very heavily affected persons, whose proportion is 
y=1—a—B8. 

Dr. Yerushalmy’s data and the discussion suggest that a diagnosis 
is always subject to error. Therefore, we have to consider the prob- 
abilities of a correct outcome of the diagnosis. These probabilities 
are different for each of the three categories of individuals distinguished 
above. Let then z and p stand for the probabilities of correct diagnosis 
for categories (i) and (ii) respectively. In consequence 1—7 will 
denote the probability of a healthy person being classified as ill or, to 
use Dr. Yerushalmy’s expression, the probability of a “false positive.”’ 
The probability of a correct diagnosis for a very ill person is postulated 
to be unity. 

It will be noticed that the above scheme is a simplification of the 
actual phenomena. In reality we may expect that the subdivision of 
human populations is much finer than is postulated here and that the 
category of “‘moderately affected” splits itself into a continuous gra- 
dation of the intensity of illness, from very slight to very heavy, with 
the probabilities of the i'lIness being detected increasing gradually 
from 1—-z to unity. As a result of this simplification, if an estimate 
of the probability p is obtained from observations, then this estimate 
must be interpreted as the estimate of the average probability of 
detecting the disease for all moderately affected persons. 

The mathematical model outlined above contains four independent 
parameters with which to describe the machinery behind figures such 
as those collected by Dr. Yerushalmy: the proportions a and 8 of the 
population and the probabilities + and p of a correct outcome of 
diagnosis. 

It may be noted that in certain cases category (iii) may be entirely 
dropped from the model with the result that the proportions a and 8 
will be postulated to add up to unity. In such cases B=1—a and, 
therefore, the model contains only three independent parameters to 
be estimated from the observations. Alternatively, it may be urgently 
needed not to ignore the gradual increase in the probability of detecting 
the disease in category (ii). Then the corresponding model will be 
more complicated. 

In order to be able to estimate all the parameters in a mathematical 
model, it is necessary to write down an equal number of independent 
equations and this implies at least an equal number of observed inde- 
pendent frequencies. Moreover, should it not only be necessary to 
estimate the unknown parameters, but also to test some hypotheses 
concerning them, the number of independent frequencies will have to 
be larger than the number of parameters. Thus, if we assume the 
above model with four parameters and desire to test some hypotheses 
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concerning it, then the necessary data would have to involve at least 
five independent frequencies. 

Let n denote the number of independent diagnoses made for each 
individual of the N individuals constituting a sample from the uni- 
verse U. Denote by k the number of positive diagnoses made for a 
particular individual. Obviously k may have one of the n+1 values 
0,1,2,...,m. Let N, stand for the number of individuals in the 
sample for which there were exactly k positive diagnoses. Dividing 
N./N=q we obtain the relative frequencies of exactly k positive 
diagnoses in the sample. There are n+1 such frequencies but only 
n of them are independent since they must add up to unity. 

It follows that, if it is desired to test some hypotheses concerning 
the four parameter model described above, the number of independent 
diagnoses made by the same method for each individual of the sample 
must be at least equal to five. If this condition is satisfied, then the 
procedure of estimating the parameters a, 8, x and p may be that of 
the best asymptotically normal estimates, explained in a recent paper.* 


(2) 


The probability that n diagnoses will give exactly k positive results 
is given by 


! 
Pr= pig pi (ea + Bp(—p)**} fork<n (1) 


and 
P,=1—a[(1—(1—2)"]—8 [1—p"] (2) 


For n=5, these probabilities must satisfy the following ec ndition 
F=20P,P2P,+4P,P2P;— P—10P,P—8P2P,=0 (3) 


To obtain the “best asymptotically normal” estimate (the BAN esti- 
mate) of the parameters, minimize the expression 
2 are (Gi— Ps)? 
x=N 24 = (4) 

under the “reduced” condition F*=0. Equating the minimizing 
values of P;, to the expressions (1) and solving for the parameters, 
the BAN estimates of these parameters are obtained. Tests of 
hypotheses concerning the parameters are based on the minimum 
values x of (4). 

The arithmetic involved in obtaining the estimates is a little long 
but in reality simple. 





*J Neyman: Contribution to the theory of x?. Proceedings of the Berkeley Symposium on Mathe- 
matical Statistics and Probability of 1945-46 (in print). 
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(i) Having the relative frequencies g, of positive diagnoses, compute 
Ag = 209294 — 1095” 
@ =49293— 169194 
a= 209094 +4993 — 392" 
A3=49192— 209095 
A4= 20q092— 891? 
a;=0 
(ii) Average the a’s with the q’s as weights and compute their 
weighted variance: 
4 


a= 10; 
=1 


4 
-—_ 2 m2 
te™= (ay — a 


=! 
(iii) Compute the “observed value” F* of F: 
F* = 20q09294+4919293— 92° — 10qoqs’— 80174 
(iv) Compute first approximations P, of the P;: 


—4a 


B.=a(1-"5 P+), i=0, eg oe 


ao 





(v) Obtain the values of 


_5P.P,—P,P, 








A=—y-s a 
5PoP,—2P/? 
_ 2P,P,—P 
2[5P.P,—2P,'| 
and solve the quadratic 
2’— Azr+B=0. 


Let z, denote the smaller and z, the larger of the two solutions. 
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Then 
ee ee 
*T4+2, “Ise, 
an P= ite (1+2,)° 


are the BAN estimates of the parameters sought. 
(vii) The final estimates of the P’s to be compared with the observed 


frequencies g, are as follows: 


P\=P,, P,=P,, 


p,=S* AP, i + BP, 


for k=2, 3, 4, and 
, - 
Py=1—2 oP. 
(viii) To test the hypothesis that the assumed scheme with four 


parameters a, 8, x and p adequately represents the machinery behind 
the observed figures, compute 


and compare this number with the tabled value of x? with one degree 
of freedom. 


(3) 


The possibility of applying this method to data collected by Dr. 
Yerushalmy depends on whether or not one agrees to consider that 
these data are diagnoses, by the same method, independently applied to 
the same individuals. 

The closest to this assumption appear to be the results of the read- 
ing of 1,256 X-ray photographs, each by five different individuals. 
The hypothesis of the sameness of the method of diagnosis implies 
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here the assumption that each of the readers is just as good a diag- 
nostician as the others. Assuming this hypothesis, the figures of 
table 1 were obtained, giving the comparison between the observed 
and the expected numbers of positive diagnoses relating to four sets 
of X-ray photographs made by four different methods. The lower 
part of the table gives the BAN estimates of the four parameters 
involved. The last line gives the values of the x?. Since the tabled 
values of x? for one degree of freedom are x3;=3.841 and x.3:=6.635, 
it is seen that observations do not suggest any serious discrepancy 
between the theory and the actual facts. , 


TABLE 1.—Observed and expected number of  serear: readings on N=1256 X-ray 
photographs, each by 5 independent readers 









































14x17 cellu. 14x17 paper 4x10 stereo 35-mm., 
k : . 
Observed | Expected | Observed | Expected | Observed | Expected | Observed] Expected 
Di cnsintdenenratiaeain 1, 125 1, 129.7 1, 123 1, 125.6 1, 095 1, 097. 4 1,071 1, 074.1 
Doce steke>aedaandwuld 47 47.6 47 47.4 63. 5 96 97.9 
ES a ee 23 18.4 19 15.8 30 26. 6 22 17.4 
ES RIT 17 20.6 18 21.1 22 25. 5 17 20. 2 
— eee eee 17 11.8 18 14.7 16 12.6 19 14.3 
SS eae ae 27 27.9 31 31.4 30 30.3 31 32.0 
ee 1, 256 1, 256. 0 1, 256 1, 256. 0 1, 256 1, 255. 9 1, 256 1, 255. 9 
a= 931 . 920 913 931 
B= 049 . 048 065 047 
A 
7= 020 032 022 022 
= 993 . 993 991 . 983 
p= 535 . 582 498 587 
a=.926, 8=.053, y=.021 
¥=.992, P=.539 
x3 2. 844 1. 596 1. 587 2. 383 

















Figures relating to the first three methods of X-ray are separated 
from the last column, because the test of the hypothesis that the 
parameters relating to these three methods are all the same failed to 
reject this hypothesis. On the other hand, the fourth method shows 
a significant difference from the others. The last two lines but one 
give the combined estimates of the parameters relating to the first 
three methods. ° 


(4) 


It will be noticed that the estimate of p, about equal to %, is small, 
perhaps unexpectedly small. However, excessive pessimism in this 
respect would be out of place. Since x appears to be very close to 
unity, a positive diagnosis must always be considered as a definite 
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warning. On the other hand, since a single negative diagnosis does 
not seem to mean very much, getting several instead of one diagnosis 
is sufficient to improve the situation. If all of them are negative, 
then it is extremely probable that the photograph concerned does not 
show signs of tuberculosis. 

Dr. Yerushalmy’s estimate of p is considerably higher than that 
obtained here. One of the reasons for this difference is that both 
estimates are estimates of the average probability of detecting moderate 
lesions. The estimates in table 1 relate to averages for all the photo- 
graphs. On the other hand, Dr. Yerushalmy’s estimates relate to 
the averages for only a part of the photographs selected in a certain 
way to avoid the inclusion of the “false positives.’”” Evidently, 
while avoiding ‘‘false positives,’’ certain light cases were also omitted. 

(5) ‘ 

Some remarks concerning the interpretation of the figures observed 
must be added. As mentioned, table 1 refers to results of repeated 
readings of the same photographs by five radiologists whom we assume 
to be equally expert and reacting to the same details visible in the 
photographs. Granting these assumptions, the estimate 


p=.539 


means that, if any one of these radiologists is given for a single reading 
one hundred photographs actually showing a ‘moderate’ degree of 
tuberculosis then we may expect about 54 positive diagnoses. In 
other words, the probabilities estimated in table 1 refer to photo- 
graphs and readers but not directly to individuals. 

Dr. Yerushalmy’s analysis suggests that while an individual may 
be moderately affected by tuberculosis, a photograph by any method 
may bear no sign of the lesions because they may be hidden by the 
ribs, ete. It is obvious that in this respect the four X-ray techniques 
may conceivably differ, and, in fact, some of them may register details 
which others do not. It is essential to realize that table 1 does not 
contain direct information on this point except through the estimates 
of a, 8 andy. To obtain more direct information on the working of the 
X-ray techniques (rather than of readers) some further data would be 
needed. These would have to include, for a number of individuals, 
at least five photographs by the same technique, each made and read 
independently from the others. The parameters a, 8 and y for such 
material would mean the proportions of individuals who have no 
sign of tuberculosis, who are moderately affected, and heavily affected. 
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The parameters + and p would mean the probabilities of correct 
diagnosis taken as a whole, including taking the photographs and 
reading them. 

There is a final remark to make, probably only an academic one. 
This is that, strictly speaking, all the parameters discussed refer to 
some agent either in the individuals photographed or on the photo- 
graphs which causes the readers to give a positive diagnosis. It is 
likely that this agent is actually tuberculosis. However, whether it is 
tuberculosis or not cannot be answered on the basis of the statistical 
data discussed here. 


DEATHS DURING WEEK ENDED SEPT. 6, 1947 


[From the Weekly Mortality Index, issued by the National Office of Vital Statistics] 








, Correspond- 
Week ended 
Sept. 6, 1947| ne week, 
Data for 93 a cities of the United States: 
2 RG RE, WS SPB ie ah eh, 7, 629 7,914 
Median for 3 prior years_- SET aE PE. eee 
Total deaths, first 36 weeks of year. LCN RAS Se aa EE 334, 532 328, 980 
Deaths under 1 year of age_. PEE Te I a 659 678 
I ncnctemnnasteaeecencsiveces« RR Ee | 
Deaths under 1 year of age, first 36 weeks of year. __..............---- - 26, 861 22, 987 
Data from industrial insurance ae 
Policies in force... nde jpdihiuienbbhinltdipescmmadcaacoun 67, 323, 407 
i inpeumawes iil 9, 139 8, 303 
Death claims per 1,000 policies in force, annual rate__...............--..- 7.1 6.4 
Death claims per 1,000 policies, first 36 weeks of year, annual rate.._____- 9.4 9.7 


























INCIDENCE OF DISEASE 


No health department, State ‘or local, can effectively prevent or control disease without 
knowledge of when, where, and under what conditions cases are occurring 


UNITED STATES 


REPORTS FROM STATES FOR WEEK ENDED SEPTEMBER 13, 1947 
Summary 

The reported incidence of poliomyelitis increased from 826 cases 
last week to 959 for the current week, as compared with a decline 
during the corresponding week last year from 1,726 to 1,620. Slight 
decreases occurred during the current week in the West North Cen- 
tral, South Central, and Mountain areas. The 16 States reporting 
currently more than 12 cases are as follows (last week’s figures in 
parentheses): Jncreases—Massachusetts 43 (34), Rhode Island 16 
(14), New York 129 (95), Pennsylvania 65 (33), Ohio 213 (195), 
Indiana 33 (28), Illinois 106 (87), Michigan 70 (45), Wisconsin 17 (15), 
Missouri 15 (5), West Virginia 17 (9), North Carolina 15 (11); ° 
decreases—Connecticut 19 (23), New Jersey 22 (34), Kentucky 13 (15), 
California 19 (21). Totals reported by geographic areas since March 
15 (the approximate average date of lowest seasonal incidence for the 
country as a whole in past years), are as follows (figures for the 
corresponding period last year in parentheses): New England 422 
(334), Middle Atlantic 858 (1,002), East North Central 1,564 (3,235), 
West North Central 532 (4,847), South Atlantic 480 (847), East South 
Central 152 (743), West South Central 218 (1,509), Mountain 196 
(1,167), Pacific 583 (1,629). The total for this period is 5,005, as 
compared with 15,313 for the same period last year, and a 5-year 
median of 7,612. 

No case of smallpox was reported during the current week. Of the 
total of 146 cases of typhoid and paratyphoid fever reported (last 
week 111, 5-year median 162), Kentucky reported 21 (including 7 
paratyphoid), Texas 14 (2 paratyphoid), and Virginia 13 (6 para- 
typhoid). Of 41 cases of infectious encephalitis, (5-year median 22), 
North Dakota reported 11, Minnesota 6, and California 5. Of 146 
cases of undulant fever, 43 occurred in Iowa, 17 in Illinois, and 12 in 
Wisconsin. 

Deaths recorded during the week in 93 large cities of the United 
States totaled 8,264, as compared with 7,629 last week, 8,607 and 
8,238, respectively, for the corresponding weeks of 1946 and 1945, 
and a 3-year (1944-46) median of 8,238. The total for the year to 
date is 342,796, as compared with 337,587 for the corresponding period 
last year. Infant deaths-in these cities totaled 735, as compared with 
659 last week and a 3-year median of 614. The total to date is 
27,596, as compared with 23,677 for the corresponding period last year. 

(1457) 
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Telegraphic morbidity reports from State health officers for the week ended Sept. 13, 
1947, and comparison with corresponding week of 1946 and 5-year median 


In these tables a zero indicates a definite report, while leaders imply that, although none was reported, 
cases May have occurred. 




















































































Meningitis, 
Diphtheria Influenza Measles a — 
. Week Week | Week Week 
Division and State | ended— Me- ended— Me- ended— Me- ended— Me. 
dian dian dian dian 
Sept. | Sept. | 1942-| Sept. | Sept. | 1942- | Sept. | Sept. | 1942- | Sept. | Sept. | 1942- 
13, 14, 46 13, 14, 46 13, 14, 46 13, 14, 46 
1947 | 1946 1947 1946 1947 1946 1947 | 1946 
NEW ENGLAND 
ESE eee 0 1 Oj... 4 35 3 1 0 1 
New Hampshire. --- 0 0 ae ae 4 1 0 0 0 
.,  . eee 0 1 = 1 7 6 0 0 0 
Massachusetts. ----- 3 4 «RGR: oMteees 21 29 29 0 0 1 
Rhode Island..-..... 1 0 0}. om 4 3 1 2 2 
Connecticut. .....- 3 3 0 1 6 8 5 0 1 1 
MIDDLE ATLANTIC 
New York.........- 7 24 10} (4) 14 11 47 67 32 3 4 i] 
New Jersey. = 2 3 i 3 3 21 9 18 0 2 2 
Pennsylvania..-_-.... 7 10 5} (@) 21) @ 42 32 26 8 6 8 
EAST NORTH CENTRAL 
| 5 6 6 2 3 2 17 35 12 1 1 3 
Se 5 3 4 2 4 4 a 4 0 2 2 
NRT 1 ll 8 1 2 2 34 14 15 0 2 8 
Michigan 3_........ 3 5 5 1 2 39 11 11 2 1 4 
Wisconsin. -_.......- 1 0 0 7 9 16 45 21 30 1 0 0 
WEST NORTH CENTRAL 
Minnesota... .....-- 15 3 4 Af 24 1 4 1 } 1 
| ES 2 0 3 $ 5 3 2 0 1 1 
SET 3 3 3 1 1 4 1 2 0 0 2 
North Dakota. -__-- 2 1 Ss ae 16 1 1 0 0 0 
South Dakota-_--.--_- 3 0 ee Se & 2 0 0 0 
Nebraska. .........- 0 2 4 - 1 2 1 0 0 0 
Pe iccccinndebes 4 10 4 1 13 9 6 0 0 1 
SOUTH ATLANTIC 
ee 0 0 0 f 2 1 0 0 0 
Maryland ?______._- 6 7 1}. Sees. 9 9 5 0 1 1 
District of Columbia 0 0 0 ol ie 5 1 i 0 0 
SS (SS 2 10 10 136 109 90) 10 12 5 0 3 3 
West Virginia....... 7 3 8 SE ee 10 2 2 2 1 1 
North Carolina... 20 10 heh Cat Sas 3 5 5 0 1 1 
South Carolina. .-._- 9 1 17 150) 39 122 & 1 6 1 0 1 
0 eS 8 is 21 1 13 13 1 7 3 0 0 0 
eal antares 4 10 5 Re 1 10 2 3 1 1 1 
EAST SOUTH CENTRAL 
Kentucky. -.......-- 2 a Se es See | Se 3 2 0 0 
Tennessee. ........- 4 5 14 5 5 5 5 2 3 3 0 3 
Alabama...........- 8 12 18 5 25 17 1 3 2 1 3 1 
Mississippi #__...... 18 14 14 ae TS =a: Soe SSN 0 2 1 
WEST SOUTH CENTRAL 
Arkansas. .......... 4 8 6 7 5 10 4 50 0 1 
Louisiana........... 1 5 6 3 8 7 ca . ;, 2 3 2 
Oklahoma. ......... 0 3 5 6 8 8 1 4 2 2 0 0 
We sccentesanses 24 35 35 183 328 284 39 14 18 6 3 2 
MOUNTAIN 
Montana... ........- 2 1 1 8 12 2 6 13 6 0 1 0 
SE 1 2 0 8 3 kan : 6 1 0 3 0 
Wyoming.........-- 0 0 0 SR PRY al 1 2 1 0 0 0 
Colorado............ 6 3 4 2 16 6 6 2 4 1 0 0 
New Mexico.......- 0 3 me ee bedi 3 1 0 0 0 
Be iitintccdneties 0 2 1 10 18 24 6 20 3 0 0 0 
| See Sere eee 1 1 «RS Se. ee 2 8 0 0 0 
Pe cisccnaneane 0 0 SS RR, SE Fee FA. SS 0 0 0 
PACIFIC 
Washington--_...... 1 5 ee ees 9 4 15 1 0 2 
I encta transected 2 5 Ren Se 3 3 17 15 2 0 1 
California........_.. _ 3) 8 = 4 3 10 48 47 49 1 5) 9 
a 210' 273) 302 546 623 657 535 487 492 41 50 93 
Ge WOR ig ii cccceced 7, 737| 10, 828| 8, 541/305, 039 194, 096) 84, 051/186, 610 641, 115 540, 807|52, 641) 4, 666/ 6, 495 
Seasonal low week 4 | (27th) July 5-11 | (30th) July 26-Aug. 1 | (35th) Aug. 30-Sept. 5| (37th) Sept. 13-19 
Total since low.....| 1,440] 2, 200| 2,054! 3,526] 3,899! 3,899 1, 1081 1,030] 1, 026/53, 613] 6, 170] 8, 947 





1 New York City only. 


2? Philadelphia only. 


3 Period ended earlier than Saturday. 


‘ Dates between-which the approximate low week ends. The specific date will vary from year to year. 


5 Correction: Arkansas 
from cumulative totals, 











y' 
, week ended August 30, meningococcus meningitis 0 cases (instead of 2). Deducted 
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Telegraphic morbidity reports from State health officers for the week ended Sept. 13, 
1947, and comparison with corresponding week of 1946 and 5-year median—Con. 

























































































Typhoid and para- 
Poliomyelitis Scarlet fever Smallpox yphold lover 
Week Week Week Week 

Division and State ended— | Me- ended— Me- ended— | Me-| ended— Me- 
dian dian pi Gee fT" | Gan 

Sept. | Sept. | 1942-| Sept. | Sept. | 1942- | Sept. | Sept. | 1942- | Sept. | Sept. | 1942- 

13, 14, 46 13, 14, 46 13, 14, 46 13, 14, 46 

1947 | 1946 1947 1946 1947 | 1946 1947*| 1946 
NEW ENGLAND 
2 2 6 2 8 0 0 0 0 0 0 
11 1 0 4 1 0 0 0 v 0 0 
1 2 0 6 3 0 0 0 0 0}. 0 
33 33 0 43 57 0 0 0 1 3 6 
4 0 3 2 2 0 0 0 0 0 1 
6 12 9 8 12) 0 0 0 0 1 1 
87 87 50 53} - 79 0 0 0 5 4 7 
16 20 13 27 22 0 0 0 4 7 4 
19 19 28 51 57 0 0 8 ll q 
55 30 49 47 64 0 2 0 3 9 7 
29 13 7 17 21 0 0 0 75 3 3 
193 66 17 32), 49 0 1 0 3 3 3 
74 29 15 32 38 0 0 0 5 4 3 
121 31 14 27 29 0 0 0 1 1 1 
187 25 ll 12 20 0 0 0 0 0 0 
42 29 2 12 13 0 0 0 4 2 1 
! 95 13 7 14 18 0 0 0 3 1 4 
North Dakota........ 3 35 2 7 0 4 0 0 0 0 0 0 
South Dakota......... 1 19 1 0 0 2 0 0 0 0 0 
i ccttidcsconen 7 37 13 1 5 7 0 0 0 0 0 
RR aE 8 64 10 6 22 22 0 0 0 0 3 3 
SOUTH ATLANTIC 
Delaware..........___- 11 0 1 2 3 1 0 0 0 0 0 0 
Maryland ?___._.._.___. 2 5 5 6 13 16 0 0 0 0 1 2 
District of Columbia. -- 3 0 2 4 2 4 0 0 0 0 0 0 
We iaicccsaccoce 10 7 7 5 26 27 0 0 0 13 1 6 
West Virginia......___. 17 7 3 9 28 48 0 1 0 3 1 fi 
North Carolina....__- 15 8 4 8 33 36 0 0 0 3 0 5 
South Carolina. -._.._-. 2 0 1 5 1 8 0 0 0 5 1 3 
ESR 3 9 4 6 5 15 0 0 0 4 1 6 
ct didantaancicscnts 5 17 6 4 4 5 0 0 0 4 0 1 
EAST SOUTH CENTRAL 
SS 13 3 5 7 14 18 0 0 0 21 2 8 
ennessee.............. 12 8 8 20 20 27 0 0 0 2 2 10 
, SS 1 10 3 2 8 23 0 0 0 1 2 4 
Mississippi*_.......... 0 14 3 6 7 8 0 0 0 3 1 5 
WEST SOUTH CENTRAL 
Arheneas........s...<-- 5 17 5 3 1 4 0 0 0 9) 3 38 
Louisiana.............. 0 21 4 3 2 5 0 0 0 1 4 6 
Oklahoma. ............ 1 11 11 6 2 3 0 0 0 4 5 5 
Fe atitttitntanin beeen 4 28 28 15 24 18 0 0 0 14 13 13 
MOUNTAIN 

ESIC TA 0 6 4 2 4 9 0 0 0 2 1 0 
Pe RRR 9 3 1 12 2 2 0 3 0 0 3 0 
Wyoming.............. 0 10 1 3 0 2 0 0 0 0 0 0 
| RS 3 65 16 8 10 10 0 0 0 0 2 1 
New Mexico........._. 5 19 2 2 3 3 0 0 0 1 2 2 
| ae hileinelnpetes 1 5 2 3 2 1 0 0 0 5 14 3 
ca, 1 7 7 4 3 5 0 0 0 0 0 0 
Se hiccenctemtanbeod 0 0 0 0 7 0 0 0 0 1 0 0 
9 40 25 ll 7 12 0 0 0 0 0 1 
6} 17 12 6 8 waftn @& a =. 3 
, 19} 153] 46) 26 82 77 V) 0 0 3 1 5 
_ SER Dee 959) 1,620) 1,020) 433) 751|_— 49} O7] S146) 14) 162 
37 weeks_........ 5, 617'15, 780| 8, 000| 64, 085| 80, 227/101, 240, 148| 286 314\72, 745| 3, 0001 3, O24 

Seasonal low week ¢....| (11th) Mar. 15-21 | (32nd) Aug.9-15 | (5th) Aug. 30- | (1th) Mar. 15-21 
Total since low.......- 5, 00518, 313| 7,612} 1, 982] 2, 9324 3, 689 i} 7| 7/72, 260 2, 525| 3, 108 








3 Period ended earlier than Saturday. 


4 Dates between which the approximate low week ends. T 
* Including paratyphoid fever re 
Indiana 1; Virginia 6; South Carol 


d separatel 
1; Florida 1; k 


entucky 7; Texas 2. 


he specific date will vary from year to year. 
as follows: New York 1; New Jersey 4; Pennsylvania 2; 


? Delayed report: Indiana, week ended August 30, typhoid fever 3 cases. Included in cumulative totals 


only. 
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Telegraphic morbidity reports from State health officers for the week ended Se . ie 
1947, and comparison with corresponding week of 1946 and 5-year median— 


Week ended Sept. 13, 1947 


En- | Rocky 
Division and State Tula- 























NEW ENGLAND 


MIDDLE ATLANTIC 
New York 














Virginia 
West Virgi 





























Same week: 1946 
Median, 1 
37 weeks: 1947 
. 1, 711)12, 515 
.| 1,353'12, 515 




















3 Period ended earlier than aetarda. 
§ 2-year average, 1945-46. 

Anthraz: New York 1, California 1. Leprosy: 

Alaska, week ended September 6, 1947: Influenza 34; 5 eeeente « septic sore throat 2. 

Territory of Hawaii, week ended September 13, 1947: Bacillary dysentery 1; whooping cough 44. 





8asBo 

qyZnoo Zuydoog 
SaSBO JOA 
proqd 43818 

pus pyoqdéy 








October 3, 1947 


seseo xod]jeurg 





Sesto 
19A0j 40[180g 





Sesto 
S}IJ[9AUMIOF[Og 





syivep 
eyuoumneug 





Best 
‘snod0003ulu 


“oul ‘sj |3ujueW 





Saseo sojsBeyy 





syj8eq 





$38BD 








sos8o ‘sn 
“Uy ‘syeqdeoug 





City reports for week ended Sept. 6, 1947 
This table lists the reports from 87 cities of more than 10,000 population distributed throughout the United 


States, and represents a cross section of the current urban incidence of the diseases included in the table. 


saswo Bey gdyq 


WEEKLY REPORTS FROM CITIES ' 








Columbus. .......... 
Fort Wayne. 

Indianapolis. _._..... 
South Bend......_... 
Terre Haute....__..- 


Cleveland -.......... 
Indiana: 


Cincinnati........... 


i Ee 


EAST NORTH CENTRAL 


Ohio: 


Pittsburgh........... 


ersey: 
SRC 
Newark....._... 
_.  x—oeenanees 
Pennsylvania: 
Philadelphia......... 


Rochester............ 


8 


New York........... 
New 


MIDDLE ATLANTIC 
Ee 


New York: 








‘In some instances the figures include nonresident cases. 


Division, State, and City 
NEW ENGLAND 


WEST NORTH CENTRAL 
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City reports for week ended Sept. 6, 1947—Continued 





Division, State, and City 


in- 


us, Cases 





halitis, 
deaths 


cases 


Diphtheria cases 
Zs 

fectlo 
Cases 
Deaths 
Measles cases 
Meningitis, men- 

ingococcus, 

Pneumonia 
Poliomyelitis 


Ence 








| 


oow 
cone 


aco 














Corresponding week, 1946* 
Average 1942-46*__....._- 
































b co ja 
> So |e 
; 23) 
4) = ee if 
A. & asae| & 
on = o 
= | 2 |eael2 
rs) J PAZ) S 
n am |e S 
0 0 0 6 
0 0 0 7 
0 0 0 1 
3 0 0 y 
0 0 0 3 
0 0 0 1 
99 0 13 | 1,023 
137 0 25 537 
219 0 28 800 














* Exclusive of Oklahoma City. 


23-year average, 1944-46. 
3 5-year median, 1942-46. 


Anthrar.—Cases: Philadelphia, 1. 


Dysentery, amebic.—Cases: New York, 5; Chicago, 1; St. Louis, 1; New Orleans, 3; Los Angeles, 1 


Dysentery, bacillary.—Cases: New York, 1; Los Angel es, 1. 

Leprosy.—Cases: New York, 1. 

Rocky Mountain spotted fever. “7 ae New York, 1; Philadelphia, 1; Columbus, 1 
Tularemia.—Cases: New York, 

Typhus fever, endemic.—Cases: "Atlante, 1; New Orleans, 3; Dallas, 1 


Rates (annual basis) per 100,000 population, by geographic groups, for the 87 cities 
in the preceding table (latest available estimated population, 34,406,100) 


























2 62 nm Zz a ¥ e* 4 
Sree as a3 2/2/38 | 2 lee |# 
° ro = e a0 Se 
fe lgea 2 | 8 lef | ez Ze|/Se| # gs.| 3 
$s =s n 3 = =3 =3 os S.— 2) wm 
sE/ES | 3 2 » |e sf} eei~é = IZsE 23 
- n - un 
a ($08) £14] lseeis 18 |S | F leesis 
& j2s2) 9) 2] = s8tl2 13 18 | & [bres 
a la 614/\s i& le 18 | 8 le IB 
New England.............| 10.5] 0.0] 0.0] 0.0 0.0| 26.3 11103] 16/ 00] 00] 242 
Middle Atlantic..__.._- 3.7/ 00] 09] 00 19/338] 21.8] 15| 00] 28] 117 
East North Central____- 1.2 0.0 1.2 0.6 0.6 | 16.6 | 88.9 17 0.0 0.0 212 
West North Central. .__- 2.0 9.9 0.0 0.0 2.0 | 33.8 | 17.9 20 0.0 0.0 286 
South Atlantic.........| 5.1] 00] 17] 00 1.7 | 32.2] 17.0 8| 60] 34] 185 
East South Central__..-- 0.0 5.9 0.0 0.0 11.8 | 20.5 5.9 12 0.0 0.0 41 
West South Central._.._| 0.0] 0.0] 5&1] 25 25|68.6| 0.0] 15] 00/127 20 
41.3 0.0 0.0 0.0 0.0 | 50.0 8.3 50 0.0 0.0 314 
Saa Seenchithonsss 6.3/ 16] 00] 00 0.0 | 2.6) 95 5| 00] 00 43 
CO 41] Li] La] 03 15|2.9/397| 15] 00] 20; 155 
| 
































PLAGUE INFECTION 


IN LOGAN COUNTY, KANSAS, 
COUNTY, WASHINGTON 


AND KITTITAS 


Plague infection has been reported proved in fleas from rodents 
collected in Logan County, Kansas, and Kittitas County, Washington, 
as follows: 
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KANSAS “i 


Logan County—Collected on August 19, a pool of 6 fleas from 5 
grasshopper mice, Onychomys sp., and a pool of 538 fleas from 62 
prairie dogs, Cynomys sp., all trapped on a ranch 2 miles north and 2 
miles west of Russell Springs. 


WASHINGTON 


Kittitas County—A pool of 76 fleas from 46 chipmunks, Hutamias 
sp., trapped on August 14 at a location 6 miles southeast of Kittitas. 


TERRITORIES AND POSSESSIONS 
Puerto Rico 


Patillas Malaria.—Under date of September 8, an outbreak of 
“pernicious malaria” was reported in Patillas, Puerto Rico, with 347 
cases and 26 deaths in August and through September 8, and additional 
cases had occurred in nearby areas. The deaths are said to have been 
in children under 5 years of age. Physicians of the Insular health 
department agree that the disease is malaria; however, blood ex- 
aminations are being made and autopsies performed to substantiate 
the diagnosis. The possibility of the introduction of a malaria 
vector is being given consideration, but. thus far no mosquito vectors 
other than the native species, Anopheles albimanus, A. grabhamii, 
and A. vestitipennis, have been found. 

United States Public Health Service officials are cooperating with 
the Insular health department in combating the epidemic; but the 
Commissioner of Health states that the outbreak could not be ex- 
pected to be brought to an end immediately as the locality is in a 
malarious area of the island. 





FOREIGN REPORTS 


BELGIUM 


Poliomyelitis —According to information dated September 5, 1947, 
110 cases of poliomyelitis with 18 deaths have been reported in the 
province of Liege, Belgium, since the beginning of the outbreak. The 
center of the outbreak is the city of Liege and suburbs, with 20 to 30 
cases reported in other provinces. 


CANADA 


Provinces—Communicable diseases—Week ended August 23, 1947.— 
During the week ended August 23, 1947, cases of certain communicable 
diseases were reported by the Dominion Bureau of Statistics of 
Canada as follows: 





British 


Sas- " 
Disease 5 Colum- 
bia 





oo 
a 


ryse , bacillary 
Enceph tis, infectious _ - 
easles 


' ' ' 
: =e : 
1 0 SrwBoRHw: -: 


Undulant fever 
Venereéal diseases: 



































GREAT BRITAIN 


England and Wales—Poliomyelitis—During the week ended August 
30, 1947, 657 cases of poliomyelitis (including polioencephalitis) were 
reported in England and Wales, making a total of 4,276 cases reported 
to date. 

(1465) 
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JAMAICA 


Notifiable diseases—4 weeks ended August 30, 1947.—During the 4 
weeks ended August 30, 1947, cases of certain notifiable diseases were 
reported in Kingston, Jamaica, and in the island outside of Kingston, 


as follows: 



































Disease Kingston a. Disease Kingston a. 
LO 2 SS SS ae Nees 1 
FORTS 1 2 || Tuberculosis (pulmonary) .--- 86 68 
Dysentery, unspecified. - -_-_- 2 ih; © | eRe 13 59 
hel Set ANTLERS, FER 3 || Typhus fever.......- ities 4 2 
Pv ddiccnaccassaice 2 1 

SCOTLAND 


Poliomyelitis —For the week ended August 30, 1947, 172 cases of 
poliomyelitis were reported in Scotland as compared with 120 cases 
reported for the preceding week. For the week ended August 23, 
1947, 33 cases of poliomyelitis were reported in the city of Glasgow. 


REPORTS OF CHOLERA, PLAGUE, SMALLPOX, TYPHUS FEVER, AND 
YELLOW FEVER RECEIVED DURING THE CURRENT WEEK 


Norte.—Except in cases of unusual incidence, only those places are included which had not previously 
reported any of the above-mentioned diseases, except yellow fever, during recent months. All reports of 


yellow fever are published currently. 
A table showing the accumulated figures for these diseases for the year to date is published in the PUBLIC 


HEALTH Reports for the last Friday in each month. 
Plague 


Belgian Congo—Costermansville Province—Beni.—On September 1, 
1947, 2 cases of plague with 1 death were reported in Beni, Coster- 


mansville Province, Belgian Congo. 
Madagascar—Mananjary.—For the week ended August 30, 1947, 


5 cases of plague were reported in Mananjary, Madagascar. 
Smallpox 


Ecuador.—Smallpox has been reported in Ecuador as follows: 
Guayaquil—week ended September 6, 1947, 47 cases including 
alastrim; Quito—June 16-30, 1947, 1 case, July 16-31, 1947, 3 cases, 
August 1-15, 1947, 26 cases, 1 death, including 15 cases reported as 


varioloid. 
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